Deformable shape models for 2D object segmentation

Given a set of images showing individual 2D instances of an object class, the goal is to learn object class deformation in 2D for segmentation automatically. Class deformation is modelled by linear combinations of basis shapes. Usually, given segmentation data and correspondences, such basis shapes can be easily learned with Principal Component Analysis. Here, we are dealing with unsegmented RGB images. We show how to learn segmentations and deformation sequentially in an iterative framework. Variations of the basic algorithm are explained, tested and compared. In order to introduce smoothness priors and data dependent pairwise terms, Graph-cut can be incorporated. The final results show that explicitly restricting segmentations by a linear subspace of shape deformation, leads to significant improvements

Investigating Sensorimotor Control in Locomotion using Robots and Mathematical Models

Locomotion is a very diverse phenomenon that results from the interactions of a body and its environment and enables a body to move from one position to another. Underlying control principles rely among others on the generation of intrinsic body movements, adaptation and synchronization of those movements with the environment, and the generation of respective reaction forces that induce locomotion. We use mathematical and physical models, namely robots, to investigate how movement patterns emerge in a specific environment, and to what extent central and peripheral mechanisms contribute to movement generation. We explore insect walking, undulatory swimming and bimodal terrestrial and aquatic locomotion. We present relevant findings that explain the prevalence of tripod gaits for fast climbing based on the outcome of an optimization procedure. We also developed new control paradigms based on local sensory pressure feedback for anguilliform swimming, which include oscillator-free and decoupled control schemes, and a new design methodology to create physical models for locomotion investigation based on a salamander-like robot. The presented work includes additional relevant contributions to robotics, specifically a new fast dynamically stable walking gait for hexapedal robots and a decentralized scheme for highly modular control of lamprey-like undulatory swimming robots

Interfacing a salamander brain with a salamander-like robot: Control of speed and direction with calcium signals from brainstem reticulospinal neurons

An important topic in designing neuroprosthetic devices for animals or patients with spinal cord injury is to find the right brain regions with which to interface the device. In vertebrates, an interesting target could be the reticulospinal (RS) neurons, which play a central role in locomotor control. These brainstem cells convey the locomotor commands to the spinal locomotor circuits that in turn generate the complex patterns of muscle contractions underlying locomotor movements. The RS neurons receive direct input from the Mesencephalic Locomotor Region (MLR), which controls locomotor initiation, maintenance, and termination, as well as locomotor speed. In addition, RS neurons convey turning commands to the spinal cord. In the context of interfacing neural networks and robotic devices, we explored in the present study whether the activity of salamander RS neurons could be used to control off-line, but in real time, locomotor speed and direction of a salamander robot. Using a salamander semi-intact preparation, we first provide evidence that stimulation of the RS cells on the left or right side evokes ipsilateral body bending, a crucial parameter involved during turning. We then identified the RS activity corresponding to these steering commands using calcium (Ca2+) imaging of RS neurons in an isolated brain preparation. Then, using a salamander robot controlled by a spinal cord model, we used the ratio of RS Ca2+ signals on left and right sides to control locomotion direction by modulating body bending. Moreover, we show that the robot locomotion speed can be controlled based on the amplitude of the Ca2+ response of RS cells, which is controlled by MLR stimulation strength as recently demonstrated in salamanders

Vortex interactions during in-line swimming in live fish: Implications for fish schooling

The 9.5th international symposium on Adaptive Motion of Animals and Machines. Ottawa，Canada (Virtual Platform). 2021-06-22/25. Adaptive Motion of Animals and Machines Organizing Committee

Adaptive Compliant Foot Design for Salamander Robots

Aquatic stepping gaits in animals arguably display higher speed performance as well as energetic efficiency compared to other gaits using the limbs (i.e walking). This suggest that the foot structure and function contributes at a great extent on the propulsive force generation. This work presents the design of a salamander foot, in which the dimensions, angle range, aspect ratios and the kinematics of different salamander species were condensed in simple parameters. The foot implementation was based in the compliant SoftHand design of Pisa/IIT, in which one motor actuates the whole foot. The prototype design parameters are scaled up from the dimensions of a Tiger salamander (Ambystoma tigrinum's) foot. The results from experiments using a motion capture system to retrieve the kinematics of the foot and the force plates to measure normal forces, allow to describe when and how each of the fingers act during the whole stride, impacting the ground reaction forces (GRFs). We attempt to provide a richer understanding in locomotion schemes of salamanders featuring robust ground placement and to make robotic platforms more accurate W.r.t. biology. Qualitative comparisons between the animal and the prototype show that the robotic foot is capable to generate a GRF pattern similar to that of the animals. As additional features, the foot also shows terrain adaptability and simultaneous high resilience to hitting obstacles during operation

Exploiting passive dynamics for robot throwing task

Throwing is a complex and highly dynamic task. Humans usually exploit passive dynamics of their limbs to optimize their movement and muscle activation. In order to approach human throwing, we developed a double pendulum robotic platform. To introduce passivity into the actuated joints, clutches were included in the drive train. In this paper, we demonstrate the advantage of exploiting passive dynamics in reducing the mechanical work. However, engaging and disengaging the clutches are done in discrete fashions. Therefore, we propose an optimization approach which can deal with such discontinuities. It is shown that properly engaging/disengaging the clutches can reduce the mechanical work of a throwing task. The result is compared to the solution of fully actuated double pendulum, both in simulation and experiment

Inverse kinematics and reflex based controller for body-limb coordination of a salamander-like robot walking on uneven terrain

Search and rescue (SAR) missions are being carried out by several types of robots. They include ground, marine and air vehicles depending on the terrain and mission to be tackled. A particular niche for SAR activities are shallow waters. They present high difficultly for conventional ground or marine robots because of the mix of water and ground. Such an environment is difficult to be accessed for a robot without some built-in amphibious capabilities. Our lab has experience in the design of amphibious salamander-like robots. In order to consider whether these robots would be suited for SAR missions in shallow waters, a key requirement is the ability to tackle rough terrains. In this paper we present a control framework for a highly redundant salamander-like robot. It involves bio-inspired spine control, inverse kinematics-based limb control, proper limb-spine coordination, reflex mechanisms and attitude control. The framework is validated in a simulation and on the real robot. In both cases, the robot is used in two different configurations: with and without its tail, in order to investigate how the tail (which is necessary for swimming) affects ground locomotion. With this exploration, we aim to set the precedent for improving the problem of dynamic locomotion of salamander-like robots over unperceived rough terrain. Our results confirm that the design of reflexes like stumbling and extension, combined with an attitude controller, allows for the improving of the performance of the robot in a generic rough terrain which includes stairs, holes and bumps with several levels of complexity adjusted according to the robot dimensions